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We  conducted  an  experiment  in  which  tactile  messages  were  created 
based  on  five  common  military  arm  and  hand  signals.  We  compared 
response  times  and  accuracy  rates  of  novice  individuals  respond¬ 
ing  to  visual  and  tactile  representations  of  these  messages,  which 
were  displayed  either  alone  or  in  congruent  or  incongruent  combi¬ 
nations.  Analyses  were  conducted  on  trials  where  tactile  and  visual 
signals  messages  were  presented  either  individually  or  concurrently. 
Results  indicated  beneficial  effects  for  concurrent,  congruent  mes¬ 
sage  presentations  with  both  modalities  showing  a  superior  response 
time  and  improved  accuracy  when  compared  to  individual  presenta¬ 
tions  in  either  modality.  These  results  confirm  the  promise  for  tactile 
messages  to  augment  visual  messaging  in  challenging  and  stressful 
environments  where  visual  messaging  may  not  always  be  possible. 


Introduction 

Many  operational  conditions  such  as  combat,  firefighting,  or  certain  law  enforce¬ 
ment  and/or  emergency  management  situations  impose  significant  demands  on 
operator’s  sensory  capabilities.  Noisy  (e.g.,  weapons  fire,  vehicle  engines,  etc.) 
and  murky  (e.g.,  smoke,  sandstorms)  environments,  for  example,  impose  great 
demands  on  hearing  and  vision,  and  can  compromise  the  ability  to  exchange  crit¬ 
ical  information  through  conventional  communication  pathways.  To  circumvent 
these  environmental  difficulties  it  may  be  possible  to  provide  a  redundant  source 
of  information  through  the  modality  of  touch,  by  using  tactile  signaling. 

Previous  studies  have  shown  tactile  systems  can  produce  relatively  stable  per¬ 
formance  improvements  across  a  variety  of  body  orientations  even  when  spatial 
translation  is  required  (Oron-Gilad,  Downs,  Gilson,  and  Hancock,  2007;  Terrence, 
Brill,  &  Gilson,  2005)  as  well  as  in  the  presence  of  physiological  stress  (Merlo, 
Stafford,  Gilson,  &  Hancock,  2006).  Most  of  human  information  processing  uses 
multiple  sensory  inputs,  such  as  the  synthesis  of  visual  and  auditory  cues  (Hancock, 
2005;  Spence  &  Driver,  2004;  Stein  &  Meredith,  1993).  Literature  on  experiments 
that  involve  the  use  of  two  modalities  of  information  presented  redundantly  each 
show  improvement  in  the  areas  of  accuracy  and  response  time  (Spence  &  Walton, 
2005;  Gray  &  Tan,  2002;  Strybel  &  Vatakis,  2004).  The  present  study  seeks  to 
show  that  similar  congruency  benefits  may  be  achieved  for  more  complex  stimuli 
presented  through  both  the  visual  and  tactile  modalities. 


417 


418 


PA.  Hancock  et  at. 


-J>  ■€>  ■<?  -<s>  -O  •0'''^l 

-^•■  7 . 

-■  W  . ' 


. ^  -  »...  .  , 

-0  -O  -o  -O  HQ 
'  . . 


Figure  1.  Three  tactile  displays  belt  assemblies  are  shown  above  along  with 

their  controller  box. 


Experimental  method 

Experimental  participants 

To  investigate  the  foregoing  proposition,  twenty  participants  (9  males  and  11 
females)  ranging  in  age  from  1 8  to  48,  with  an  average  age  of  25  years,  volunteered 
to  participate.  Each  participant  self-reported  no  surgeries,  significant  scarring  or 
any  impediment  that  might  cause  lack  of  feeling  in  the  abdomen  or  torso  area, 
Additionally,  none  of  the  participants  had  any  prior  experience  with  the  presented 
arm  and  hand  signals  nor  the  tactile  signals  in  general. 


Experimental  materials  and  apparatus 

The  vibrotactile  actuators  (tactors)  in  our  system  are  model  C2,  manufactured  by 
Engineering  Acoustics,  Inc.  They  are  acoustic  transducers  that  displace  200-300  Hz 
sinusoidal  vibrations  onto  the  skin.  Their  17  gm  mass  is  sufficient  for  activating 
the  skin’s  tactile  receptors.  The  C2’s  contactor  is  7  mm,  with  a  1  mm  gap  separating 
it  from  the  tactor  aluminum  housing.  The  C2  is  a  tuned  device,  meaning  it  oper¬ 
ates  well  only  within  a  very  restricted  frequency  range,  in  this  case  approximately 
250  Hz.  The  tactile  display  itself  is  a  belt  like  device  with  eight  vibrotactile  actuators, 
an  example  of  which  is  shown  in  Figure  1 .  The  belt  itself  is  made  of  elastic  and  high 
quality  cloth  similar  to  the  material  used  by  professional  cyclists.  When  stretched 
around  the  body  and  fastened,  the  wearer  has  an  actuator  over  the  umbilicus  and 
one  centered  over  his  spine  in  the  back.  The  other  six  actuators  are  equally  spaced, 
three  on  each  side,  for  a  total  of  eight  (see  Cholewiak,  Brill,  &  Schwab,  2004). 

The  tactors  are  operated  using  a  Tactor  Control  Unit  (TCU)  which  is  a  computer- 
controlled  driver/amplifier  system  that  switches  each  tactor  on  and  off  as  required. 
This  device  is  shown  on  the  left  side  of  the  tactile  displays  belts  in  Figure  1 .  The  TCU 
weighs  1 .2  lbs  independent  of  its  power  source  and  is  approximately  one  inch  thick. 
This  device  cormects  to  a  power  source  with  one  cable  and  to  the  display  belt  with 
the  other  and  uses  Bluetooth  technology  to  communicate  with  the  computer  driven 
interface.  Tactile  messages  were  created  using  five  standard  Army  and  Marine 


Crossmodal  congruency  benefits  for  tactile  and  visual  signaling  419 


File  £dc  View  Operate  Window 


Figure  2.  A  computer  screen  shot  showing  what  the  participant  viewed  as 
the  signals  were  presented.  The  participant  mouse  clicked  on  the  appropriate 
signal  name  after  each  presentation. 


Corps  arm  and  hand  signals  (Department  of  the  Army,  1987).  The  four  signals 
chosen  for  the  experiment  were,  “Attention”,  “Halt”,  “Rally”,  “Move  Out”,  and 
“Nuclear  Biological  Chemical  event  (NBC)”.  The  tactile  representations  of  these 
signals  were  designed  in  a  collaborative  effort  of  scientists  at  the  University  of 
Central  Florida  and  a  consultant  group  of  subject  matter  experts  (SMEs)  consisting 
of  former  US  Soldiers  and  Marines. 

Short  video  clips  of  a  soldier  performing  the  five  arm  and  hand  signals  were  edited 
to  create  the  visual  stimuli.  Careful  editing  ensured  the  timing  of  the  arm  and  hand 
signals  closely  matched  that  of  the  tactile  presentations  (see  Figure  2).  A  Samsung 
Q1  Ultra  Mobile  computer  using  an  Intel  Celeron  M  ULV  (900  MHz)  processor 
with  a  7"  WVGA  (800  x  480)  liquid  crystal  display  was  used  to  present  videos 
of  the  soldier  performing  the  arm  and  hand  signals.  This  computer  ran  a  custom 
Lab  VIEW  (8.2;  National  Instruments)  application  that  presented  the  tactile  signals 
via  Bluetooth  to  the  tactor  controller  board  and  captured  all  of  the  participant’s 
responses  via  mouse  input.  Participants  wore  sound  dampening  headphones  with  a 
reduction  rating  of  1 1.3  dB  at  250  Hz  to  reduce  the  effects  of  any  auditory  stimuli 
emanated  by  the  tactor  actuation. 

The  display  of  each  message  or  signal  was  presented  in  one  of  four  ways: 

•  visual  only  (video  presentation  of  the  arm  and  hand  signal) 

•  tactile  only  (tactile  presentation  of  the  arm  and  hand  signal) 

•  both  visual  and  tactile  simultaneous  and  congruent  (i.e.  the  same  signals  were 
presented  both  through  the  video  and  through  the  tactile  system) 

•  Both  visual  and  tactile  simultaneous  and  incongruent  (i.e.  the  visually  presented 
signal  did  not  match  the  presented  tactile  signal). 


Experimental  design  and  procedure 

Participants  first  completed  a  computer-based  tutorial  that  described  each  arm 
and  hand  signal  individually.  For  each  signal,  a  short  description  was  presented. 
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Participants  then  viewed  a  video  of  a  soldier  performing  the  signal  followed  by 
experiencing  its  tactile  equivalent.  Finally,  the  participants  were  able  to  play  the  sig. 
nals  concurrently  (both  visual  and  tactile  representation).  Participants  were  allowed 
to  repeat  the  presentation  (i.e.,  visual,  tactile,  visual-tactile)  as  many  times  as 
desired.  Once  the  participant  reviewed  the  five  signals  in  the  two  presentati® 
styles,  a  validation  exercise  was  performed.  Participants  had  to  correctly  identify 
each  signal  twice  before  the  computer  would  prompt  the  experimenter  that  the 
participant  was  ready  to  begin. 

Each  participant  performed  two,  60  trial  blocks.  The  blocks  had  two  of  each 
signal  presented  only  visually  ( 1 0  total),  two  of  each  signal  with  only  tactile  signals 
(10  total),  four  of  each  signal  performed  simultaneously  with  both  congruent  visual 
and  tactile  presentation  (20)  and  four  of  each  tactile  and  visual  signal  performed 
simultaneously  but  incongruent  (20).  Each  participant  performed  two  blocks  oftri- 
als,  with  the  60  trials  within  the  blocks  completely  randomized  for  each  participant. 
The  entire  experiment  took  less  than  an  hour  to  complete. 


Results 

All  analyses  reported  were  conducted  using  SPSS  11.5  for  Windows  with  the  alpha  i 
level  set  at  .05  for  a  two-tailed  t-test  conducted  unless  otherwise  noted.  Resultswere 
analyzed  in  terms  of  the  speed  of  the  response  and  the  accuracy  of  the  response  under 
the  respective  conditions.  In  the  interest  of  brevity,  only  the  results  from  the  three 
conditions  of  visual  presentation,  tactile  presentation  and  congruent,  concurrent 
presentation  are  presented  in  this  work. 

A  one-way  Analysis  of  Variance  (ANOVA)  was  performed  on  the  mean  response 
times  across  the  three  experimental  conditions  of  visual  presentation,  tactile  i 
presentation  or  visual-tactile  concurrent  and  congruent  presentation,  with  the  fol-  | 
lowing  results:  F{\,  19)  ==473.45,  ,p<.01,  (r]j-.96l,  P  =  1.00).  Subsequent 
a  priori  pairwise  analysis  showed,  simultaneously  presented  congruent  signals 
resulted  in  significantly  faster  response  times  than  visual  signals  presented  alone 
;(19)  =  — 2.25,  p<.04,  see  Figure  3.  Also,  as  is  evident,  the  congruent  signals  | 
were  faster  than  tactile  alone  r(19)  =  —3.9S,p  <  .01.  Additionally,  the  visual  only 
presentation  of  the  signal  was  significantly  faster  than  the  tactile  only  presentation 
of  the  signal  t(19)  =  — 2.16,/>  <  .04.  | 

Although,  there  was  no  significant  difference  in  the  accuracy  rates  observed 
between  the  visual  and  tactile  signals  to  each  other  when  presented  alone 
r(19)  =  2.00,  p  <  .06.  There  was  a  significant  difference  in  the  performance  rates 
when  the  tactile  modality  was  compared  to  the  concurrent  congment  presentation 
of  the  signals,  t(19)  =  4.03,y;  <  .01.  The  overall  lower  accuracy  rate  for  the  tactile 
signaling  was  due  to  an  apparent  confusion  between  the  tactile  signal  for  “NBC”  and 
“Halt”,  which  have  similar  tactile  characteristics  but  low  visual  similarity.  Analysis 
without  the  “NBC”  tactile  signal  data  removes  these  differences  (it  was  not  signifi¬ 
cantly  different  with  it  included)  in  the  error  rate  between  visual  and  tactile  signals, 
while  not  influencing  the  main  effect  between  congruent  modality  signaling  and 
single  modality  signaling  at  all. 
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Condition 

Figure  3.  Response  Time  in  milliseconds  by  signal  presentation  condition. 


Discussion 

The  overall  high  accuracy  rate  displayed  by  the  participants  (over  80%  in  all  modal¬ 
ities  with  fewer  than  ten  minutes  of  training)  is  highly  encouraging  to  the  current 
form  of  tactile  display.  The  accuracy  of  the  messages  and  the  reported  inmitiveness 
with  which  they  were  received  is  also  a  testament  to  the  utility  of  the  subject  matter 
expert  information  and  the  present  tactile  “language”  transformation  format.  Sim¬ 
ilarities  in  both  tactile  and  visual  signals  that  cause  confusion  among  signals  are 
virtually  eliminated  in  concurrent  presentation.  The  rich  multi-modal  information 
format  seemed  to  produce  faster  and  more  accurate  performance. 

The  question  of  learning  complex  tactile  communication  signals,  especially  for 
use  in  adverse  or  unusual  circumstances  is  liable  to  be  an  important  future  issue. 
The  tactile  system  acts  as  a  redundancy  gain  as  the  participants  now  have  two  means 
of  receiving  communication  because  the  visual  hand  and  arm  signals  would  still  be 
available.  While  initial  testing  seems  to  result  in  superior  performance  for  tactile 
communication  and  traditional  arm  and  hand  signals  combined,  the  challenge  of  a 
universal  input  device  remains  a  significant  hurdle.  Stimulus  response  compatibility 
will  have  to  be  analyzed  carefully  to  maximize  performance  as  different  types  of 
inputs  are  considered  for  use  with  tactile  displays.  However,  when  individuals 
are  faced  with  extreme  challenges  and  the  traditional  sources  of  information  are 
either  diminished  of  eliminated  altogether,  the  tactile  system  provides  an  important 
alternative  communication  channel  and  one  that  can  and  should  be  exploited. 
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